fix(schema): emit INDEXMISSING after the vector attribute block by RaphaelFakhri · Pull Request #771 · redis/redis-vl-python

RaphaelFakhri · 2026-10-17T21:24:17Z

Motivation

With index_missing: True on a vector field, SearchIndex.create() fails. field_data put "INDEXMISSING": True inside the vector attribute block, so redis-py raises Invalid input of type: 'bool', and the server rejects the token there anyway (Bad arguments for algorithm HNSW: INDEXMISSING). FT.CREATE expects INDEXMISSING after the attribute block, which is where redis-py places it for its index_missing keyword. The review on #742 described this failure.

Fixes #770.

Changes

redisvl/schema/fields.py: drop INDEXMISSING from BaseVectorFieldAttributes.field_data, and pass index_missing=self.attrs.index_missing to the redis-py VectorField in FlatVectorField, HNSWVectorField and SVSVectorField.as_redis_field(). This matches how the other field types handle the modifier.
tests/unit/test_fields.py: replace the test that asserted INDEXMISSING in field_data with a parametrized test (FLAT, HNSW, SVS-VAMANA) that checks INDEXMISSING follows the attribute block in redis_args().
tests/integration/test_field_modifier_ordering_integration.py: create FLAT and HNSW indexes with a vector index_missing field and check that ismissing(@embedding) returns only the document without the vector.

from_existing still doesn't recover index_missing on vector fields (see the comment in tests/unit/test_convert_index_info.py). That can follow separately now that the field can be recreated.

Release Notes

Vector fields declared with index_missing: true can now be created, and ismissing(@field) queries work on them.

Review record

This change was written with Claude Code (Claude Opus 5.5) and reviewed by a human before submission.

Reviewed at c5213f9 by Claude Code (Claude Opus 5.5) with pre-pr-review.
Lenses: correctness (always); tests (adds unit and integration tests); Redis behaviour (changes where FT.CREATE gets INDEXMISSING). Skipped: security (no query-building, MCP or destructive paths); design (small fix, no API change); docs (no docs or docstrings change).

3 findings raised, 0 kept after challenge.
Fixed: none.
Rejected: removing INDEXMISSING from field_data could break a caller that reads it (no caller outside fields.py reads field_data); the index_missing keyword might not exist on the redis-py floor (redis-py 6.3.0 accepts it and emits INDEXMISSING after the block, checked directly); SVS-VAMANA has no integration case (the unit test covers its argument order, and the code path is shared with FLAT and HNSW).
Open for the reviewer: none.

Verified: pytest tests/unit/test_fields.py tests/integration/test_field_modifier_ordering_integration.py tests/unit/test_convert_index_info.py (85 passed, Redis 8.4); same two test files with fields.py from main (5 failed, the new cases); pytest tests/unit --ignore=tests/unit/test_mcp (1345 passed, 1 skipped); make check-types (no issues); black and isort --profile black on the changed files (no changes).
Docs consulted: https://redis.io/docs/latest/commands/ft.create/
Not checked: cluster mode; SVS-VAMANA against a live server; Redis versions other than 8.4.

Note

Medium Risk
Changes index creation and schema recovery for vector fields; incorrect FT.CREATE ordering previously broke create(), but round-trip behavior for existing indexes now differs slightly.

Overview
Fixes SearchIndex.create() failures when vector fields use index_missing: true. INDEXMISSING was incorrectly placed inside the vector field_data block; it is now passed via redis-py’s index_missing keyword on RedisVectorField (FLAT, HNSW, SVS-VAMANA), matching lexical fields and FT.CREATE argument order.

convert_index_info_to_schema now parses INDEXMISSING from vector FT.INFO output (RESP2 list and redis-py 8 flags) and sets index_missing: true on recovered schemas, so from_existing round-trips the flag.

Tests assert INDEXMISSING appears after the attribute count in redis_args(), extend ismissing() coverage to HNSW vectors, and add index_missing on vector fields in from_existing integration cases.

^{Reviewed by Cursor Bugbot for commit 064fd78. Bugbot is set up for automated code reviews on this repo. Configure here.}

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

vishal-bala · 2026-10-19T08:04:07Z

Hi, thanks for your contribution! Can you rebase onto the latest origin/main and follow the new contribution process?

A vector field with index_missing=True put INDEXMISSING into the algorithm attribute dict, so create() failed with "Invalid input of type: 'bool'", and the server rejects the token inside the attribute block anyway ("Bad arguments for algorithm"). Pass index_missing to the redis-py VectorField instead, which appends INDEXMISSING after the block as FT.CREATE expects.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

RaphaelFakhri · 2026-10-22T16:08:53Z

Rebased onto the latest main, and the description now follows the new template with a Review record for c5213f9.

vishal-bala

Thanks for the contribution! This looks good, just a couple minor points and one additional spot for scope

vishal-bala · 2026-10-27T14:09:18Z

+            self.attrs.algorithm,
+            field_data,
+            as_name=as_name,
+            index_missing=self.attrs.index_missing,


Now that a vector field can be created with index_missing, from_existing should recover it too. Otherwise a from_existing → create(overwrite=True) round trip still quietly drops the modifier. The parser skips the trailing INDEXMISSING in the vector branch of convert_index_info_to_schema (redisvl/redis/connection.py, the flags loop around line 408), and tests/unit/test_convert_index_info.py:277-279 pins that, with a comment that's now out of date ("can't be recreated with it yet"). Could you pick it up there, flip that assertion to expect index_missing: True, and check it against the four client configs in test_client_from_existing_and_batch_search?

Done in 064fd78. parse_vector_attrs now sets index_missing: True when INDEXMISSING follows the vector params. That covers the trailing token in the list response and flags in the dict responses, including the packed RESP2 form. The assertion in test_convert_index_info.py now expects index_missing: True, and the outdated comment is gone.

The vector field in test_client_from_existing_and_batch_search (sync and async) now sets index_missing: True, so its reopened.schema == index.schema check covers the round trip. All four client configs pass against redis:8.4 with redis-py 8.1. Without the parser change, all four fail.

vishal-bala · 2026-10-27T14:09:18Z

-    field_data = flat_field_with_missing.attrs.field_data
-    assert "INDEXMISSING" in field_data
-    assert field_data["INDEXMISSING"] == True
+    assert "INDEXMISSING" not in field.attrs.field_data


Two trims: this field_data assertion checks an internal detail that the redis_args() assertions below already cover, and the "without index_missing" half at the end passed before the fix. While you're here, test_vector_fields_redis_creation_with_index_missing (line 359) only checks isinstance and .name and passed while the bug was live, so this test replaces it.

Both trims are done, and test_vector_fields_redis_creation_with_index_missing is removed in 064fd78.

vishal-bala · 2026-10-27T14:09:18Z

        index.delete(drop=True)

+    @pytest.mark.parametrize("algorithm", ["flat", "hnsw"])
+    def test_vector_field_index_missing(self, client, redis_url, worker_id, algorithm):


The unit test already covers all three algorithms, so this only needs to prove the server accepts the modifier and answers ismissing. Could you fold it into test_indexmissing_enables_ismissing_query (line 210) as a second parametrised case (text, plus vector with HNSW) instead of adding a new test?

Folded into test_indexmissing_enables_ismissing_query as text and vector-hnsw cases in 064fd78. The separate test is removed.

A vector field can now be created with index_missing, so read the INDEXMISSING modifier back from FT.INFO too. Otherwise a from_existing and create(overwrite=True) round trip drops it.

Copilot

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.

Copilot AI balanced review requested due to automatic review settings October 17, 2026 21:24

Copilot AI reviewed Oct 17, 2026

Copilot AI balanced review requested due to automatic review settings October 22, 2026 16:08

RaphaelFakhri force-pushed the fix-vector-index-missing branch from 723e8b1 to c5213f9 Compare October 22, 2026 16:08

Copilot AI reviewed Oct 22, 2026

vishal-bala self-assigned this Oct 27, 2026

vishal-bala self-requested a review October 27, 2026 13:50

vishal-bala added the auto:patch Increment the patch version when merged label Oct 27, 2026

vishal-bala requested changes Oct 27, 2026

View reviewed changes

vishal-bala mentioned this pull request Oct 27, 2026

MCP: allow locked exists filters on vector fields declared with index_missing #788

Open

fix(schema): recover INDEXMISSING on vector fields in from_existing

064fd78

A vector field can now be created with index_missing, so read the INDEXMISSING modifier back from FT.INFO too. Otherwise a from_existing and create(overwrite=True) round trip drops it.

Copilot AI balanced review requested due to automatic review settings October 28, 2026 14:48

Copilot AI reviewed Oct 28, 2026

Navigation Menu

Repository navigation

fix(schema): emit INDEXMISSING after the vector attribute block - #771

fix(schema): emit INDEXMISSING after the vector attribute block#771
RaphaelFakhri wants to merge 2 commits into
redis:mainfrom
RaphaelFakhri:fix-vector-index-missing

RaphaelFakhri commented Oct 17, 2026 •

edited by cursor Bot

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

vishal-bala commented Oct 19, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

RaphaelFakhri commented Oct 22, 2026

Uh oh!

vishal-bala left a comment

Uh oh!

vishal-bala Oct 27, 2026

Uh oh!

RaphaelFakhri Oct 28, 2026

Uh oh!

vishal-bala Oct 27, 2026

Uh oh!

RaphaelFakhri Oct 28, 2026

Uh oh!

vishal-bala Oct 27, 2026

Uh oh!

RaphaelFakhri Oct 28, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

RaphaelFakhri commented Oct 17, 2026 • edited by cursor Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Changes

Release Notes

Review record

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

vishal-bala commented Oct 19, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

RaphaelFakhri commented Oct 22, 2026

Uh oh!

vishal-bala left a comment

Choose a reason for hiding this comment

Uh oh!

vishal-bala Oct 27, 2026

Choose a reason for hiding this comment

Uh oh!

RaphaelFakhri Oct 28, 2026

Choose a reason for hiding this comment

Uh oh!

vishal-bala Oct 27, 2026

Choose a reason for hiding this comment

Uh oh!

RaphaelFakhri Oct 28, 2026

Choose a reason for hiding this comment

Uh oh!

vishal-bala Oct 27, 2026

Choose a reason for hiding this comment

Uh oh!

RaphaelFakhri Oct 28, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

RaphaelFakhri commented Oct 17, 2026 •

edited by cursor Bot

Loading